Multiple resolution video compression

ABSTRACT

An apparatus and method for compressing multiple resolution versions of a video signal are disclosed. A first resolution version of a video signal is applied to an input of a first video compressor and to an input of a video scaler. The first video compressor encodes the first resolution version of the video signal to generate a first compressed video bit stream. The video scaler generates a reduced resolution version of the video signal from the first resolution version. The reduced resolution version is supplied to a second video compressor and to the first video compressor. The first video compressor utilizes the reduced resolution version of the video signal in performing a hierarchical motion estimation (ME) search as part of the encoding process for the first resolution version. The second video compressor encodes the reduced resolution version to generate a second compressed bit stream. The second video compressor receives motion vectors or other results of the hierarchical ME search performed in the first video compressor, and uses these results to facilitate the encoding of the reduced resolution version. The apparatus and method may be used in a non-linear video editor, a video server or other video processing system. The video scaler and first and second video compressors may share memory, a transform unit and other processing hardware such that system cost and complexity are reduced.

This application is a divisional of Ser. No. 08/999,763, filed Jul. 22,1997, now U.S. Pat. No. 6,005,621.

FIELD OF THE INVENTION

The present invention relates generally to digital video compression andmore particularly to video compression methods and apparatus forproducing different resolution compressed versions of a given videosignal.

BACKGROUND OF THE INVENTION

FIG. 1 shows an “on-line” “non-linear” video editing system 10. One ormore linear video sources 12, such as video tape recorders (VTRs), eachoutput an analog or digital video signal to a compressor 14. Thecompressor 14 compresses each video signal and stores the digitalcompressed video signals in a non-linear storage device 16, such as amagnetic disk. Unlike the linear video signal source 12, which accessesvideo signals sequentially, the non-linear storage device 16 supportsrandom access of video signals. Compression of the video signals in thecompressor 14 may be achieved according to the MPEG, MPEG-2, Dolby™AC-3, etc., standards for compressing video and associated audio. SeeISO/IEC IS 13818-1, 2, 3: Generic Coding of Moving Pictures andAssociated Audio: Systems, Video and Audio. The compressed video signalsare then edited using a computer 18 which accesses the compressed videosignals for display and editing. In displaying a compressed videosignal, the non-linear storage device 16 outputs a compressed videosignal to a video decompressor 19, which decompresses the video signalfor display on a display monitor. After editing, the edited compressedvideo signal may be retrieved from the non-linear storage device 16 anddecompressed by decompressor 19 for storage on a linear storage device(such as a VTR) or transmitted.

The non-linear video editing system 10 enables the operator to accessthe video in a random fashion instead of a sequential fashion. Thisfacilitates the editing of the video and provides for rather preciseediting. The editing is said to be “on-line” in that the final editedvideo event is incrementally constructed as the edits are made. Forexample, if the operator chooses to splice a first video clip to asecond video clip, then the compressed form of the first video clip isspliced to the compressed form of the second video clip, at the time theoperator makes such a choice to produce an edited compressed videosignal. The edited video event is formed by decompressing the editedcompressed video signal.

FIG. 2 shows a an “off-line” “non-linear” video editing system 50.Multiple linear video sources 12 are used to input video signals to avideo compressor 14. The video compressor 14 compresses each videosignal and stores the compressed video signal on a non-linear storagedevice 16. As before, a computer 18 can access the compressed videosignals on the non-linear storage device 16 for viewing and editing. Thecompressed video signals are decompressed in decompressor 19 prior toviewing on a display monitor (not shown).

Unlike the video editing system 10, the video editing system 50 does notincrementally form the final edited video event as each edit decision ismade. Rather, the formation of the final edited video event is deferreduntil the editing of the video signals is complete. The computer 18generates a “edit decision list” as each editing operation is performedby the operator. Each edit decision on the edit decision list may, forexample, indicate the kind of edit transition/operation to use (cut,fade, wipe dissolve, over-dub, etc.), which video clips are operated onby the edit operation and the edit points (e.g., specific pictures oraudio frames at which the editing operation begins and ends) withinthose clips. The operator can preview a “facsimile” of the final eventas formed by performing the edit operations indicated on the editdecision list on the compressed video signals stored in the non-linearstorage device 16. After the operator has approved the edit decisions,the computer 18 accesses the original video signals stored on the linearstorage devices 12 and performs edit operations on these video signalsaccording to the edit decision list.

Typically, an off-line video editor 50 produces a final edited videoevent with better video and audio quality than an on-line video editor10. This is because the video event produced by on-line video editingcontains compression artifacts produced by the lossy or imperfectcompression and decompression of the video. Nevertheless, an on-linevideo editor 10 is less expensive than an off-line video editor 50because the on-line video editor 10 needs only a single linear storagedevice 12. Furthermore, the linear storage device 12 used in the on-linevideo editor 10 need not be as sophisticated as the linear storagedevice 12 used in the off-line video editor 50 since it does not have toadvance to multiple specific edit points (as would be necessary in theoff-line video editor in forming the final edited video event from theedit list). The on-line video editor 10 is easier to use because thephysical media (video tapes) on which the original video information isstored need not be resident in the linear storage device 12 to createthe final edited video event This is because once the video signals aretransferred to the non-linear storage device 16, the video signals neednot be retrieved from the linear storage device again.

In order to reduce the level of compression artifacts present in thefinal edited video event in the on-line video editor 10, the on-linevideo editor 10 may typically use high compression data rates, highresolution and/or inter-frame coding. Video signals from source 12 insystem 10 are compressed before being stored in the storage device 16 inorder to save disk space and because the throughput of inexpensive diskdrives is generally insufficient to support uncompressed high resolutionvideo. To insure sufficiently high quality edited video, a non-lineireditor typically uses high resolutions such as 720×480 pixels at 30frames/sec and high compressed bit rates such as 18-50 Mbits/sec.

The use of high resolutions, high bit rates and/or inter-frame encodingin the compressor 14 can increase the difficulty of processing functionssuch as accessing stored compressed video streams, playing back morethan one bit stream at the same time, and decoding/decompressing withtrick modes such as fast forward and fast reverse. A compression systemwhich utilizes compressed video bit streams having low resolution, lowbit rate and/or only intra-frame encoding does not suffer thesedrawbacks. It is therefore desirable in many applications to provide asystem in which multiple resolution and/or multiple bit rate versions ofa given video signal can be compressed and stored. The high resolutions,high bit rates and inter-frame encoding can then be utilized whennecessary, while the advantages of low resolution, low bit rates andintra-frame encoding can also be provided in appropriate applications.

Video servers represent another application in which storage of multipleversions of compressed video bit streams is desirable. Such videoservers are used to deliver video bit streams to end users over datacommunication networks. For example, a World Wide Web server may be usedto deliver video bit streams to different end users over different typesof lines, including plain old telephone service (POTS) lines, integratedservices digital network (ISDN) lines, T1 lines and the like. A versionof a given compressed bit stream that may be suitable for a POTS userwould be considered poor quality by a T1 user, and a bit stream suitablefor a T1 user would be at too high a bit rate for a POTS user. It istherefore desirable for the video server to store a given video bitstream at multiple bit rates. The “optimal” resolution for a compressedvideo bit stream is the one that yields the best subjective videoquality after decompression. This optimal resolution generally decreaseswith bit rate, such that it is desirable for the video server tocompress the different bit rate streams at different resolutions.

FIG. 3 shows a conventional video compression system 20 which generatesand stores multiple resolution versions of a given bit stream. Thesystem 20 includes a video source 12, video compressor 14 and storagedevice 16 which operate in the manner previously described in toconjunction with FIGS. 1 and 2. The system 20 also includes a videoscaler 22 which receives a given video signal from the source 12 andgenerates a number of reduced resolution versions thereof These reducedresolution versions are supplied to the video compressor 14, whichgenerates a compressed video bit stream at an appropriate bit rate foreach of the reduced resolution versions of the video signal, and storesthe resulting compressed streams on the storage device 16.

The system 20 suffers from a number of significant problems. Forexample, each of the reduced resolution versions of a given video signalare separately and independently compressed. The information used toencode the video signal at one resolution is generally not used tofacilitate the encoding process for the other reduced resolutionversions. In addition, the video scaler 22 and video compressor 14 arenot configured in a manner which enables them to share at least aportion of a common memory. The system 20 therefore requires relativelylarge amounts of memory. These and other drawbacks of the system 20unduly increase its cost and complexity, and limit its usefulness innon-linear editor applications, video server applications and numerousother important video processing applications.

As is apparent from the above, there is a need for an improved multipleresolution video compression system in which hardware and processingresources can be shared to thereby significantly reduce the cost andcomplexity of the system.

SUMMARY OF THE INVENTION

The present invention provides an apparatus and method for compressingmultiple resolution versions of a video signal. Each of the compressedversion of the video signal may be independently decompressed, i.e.,without resort to information contained in another version of the videosignal.

Illustratively, such multiple compressed versions of the video signalmay be used in a non-linear video editing system. One compressed versionof the video signal may be used to determine how to edit the video toproduce a final edited video event. A second compressed version of thevideo signal may be used to create the final edited video event. Forexample, a lower quality compressed version of the video signal may beviewed and used as a basis to form an edit decision list. After the editdecision list is complete, the actual editing operations may beperformed on a higher quality compressed version of the video signal toproduce a final edited high quality video event.

The invention permits the sharing of memory between a video scaler andvideo compressor in a video compression system, and the sharing ofmotion estimation, discrete cosine transform and other compressionhardware between multiple video compressors in the video compressionsystem. The invention thereby considerably reduces the cost andcomplexity of a video compression system. The invention alsosignificantly improves the performance and capabilities of non-linearvideo editors, video servers and other video processing applicationswhich utilize multiple resolution compression systems.

An exemplary embodiment of the invention includes first and second videocompressors and a video scaler. A first resolution version of a videosignal is applied to an input of a first video compressor and to aninput of a video scaler. The first resolution version may be a CCIR601image sequence or other high resolution unscaled video image sequence.The first video compressor encodes the first resolution version of thevideo signal to generate a first compressed video bit stream inaccordance with MPEG-2 or another suitable video encoding standard. Thevideo scaler generates a reduced resolution version of the video signalfrom the first resolution version. The reduced resolution version may bea QQCIF image sequence, a QCIF image sequence or a CIF image sequence,the images of which are {fraction (1/64)} size, {fraction (1/16)} sizeand ¼ size, respectively, relative to a full resolution CCIR601 image.Other types of images and image resolutions may also be used, and thevideo scaler may generate multiple reduced resolution versions of thevideo signal. The reduced resolution version or versions are supplied toa second video compressor and to the first video compressor. The firstvideo compressor utilizes the reduced resolution versions of the videosignal in performing a hierarchical motion estimation (ME) search aspart of the encoding process for the first resolution version. Thesecond video compressor encodes the reduced resolution version togenerate a second compressed bit stream. The second video compressorreceives motion vectors or other results of the hierarchical ME searchperformed in the first video compressor, and uses these results tofacilitate the encoding of hie reduced resolution version. Although somesharing of encoding information may occur during the compression, eachvideo signal is independently decompressible without resort toinformation contained in the other video signal.

In accordance with other aspects of the invention, the video scaler andfirst video compressor may share a common memory for temporary storageof reduced resolution images, and the first and second compressors maybe configured to utilize common motion estimation and discrete cosinetransform hardware. For example, a single discrete cosine transform unitmay be shared in a time multiplexed manner between the first and secondvideo compressors.

According to another embodiment, a nonlinear editor is provided withfirst and second video compressors. The first video compressor is forcompressing a first version of an input video signal to produce a lowquality compressed video signal. The second video compressor is forcompressing a second version of the same input video signal to produce ahigh quality compressed video signal, having a higher quality than thelow quality compressed video signal. Each of the low quality and highquality compressed video signals produced by the first and second videocompressors, respectively, are independently decompressible.

Illustratively, the low and high quality compressed video signals can beachieved a number of ways such as providing different quality first andsecond versions of the input video signal for input to the first andsecond video compressors. For example, the first version may be one ormore of the following: spatially scaled, temporally scaled,non-interlaced (progressive scanned or de-interlaced), or bit ratereduced, whereas the second version may be one or more of the following:full spatial resolution, full temporal resolution, interlaced, or fullbit-rate. Alternatively, or in addition, the first and second videocompressors can compress the first and second versions of the inputvideo signal differently. For example, the first compressor nay use oneor more of the following: a less efficient compression standard, such asMPEG-1, intra-coding only, frame prediction only, or low bit rate codingwhereas the second compressor may use one or more of the following: ahighly efficient compression standard, such as MPEG-2, inter andintra-coding, field and frame prediction, or high bit rate coding.

Illustratively, the non-linear video editor includes a decoder which cansimultaneously present more low quality compressed video signals thanhigh quality compressed video signals in real time. Alternatively, or inaddition, the non-linear video editor includes a computer that canperform more edits/effects (e.g., cuts, wipes, fades, dissolves, trickmodes, etc.) on the low quality compressed video signals than on thehigh quality compressed video signals.

These and other features and advantages of the present invention willbecome more apparent from the accompanying drawings and the followingdetailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a conventional off-line video editing system.

FIG. 2 shows a conventional on-line video editing system.

FIG. 3 shows a conventional video compression system.

FIG. 4 shows an exemplary video compression system suitable forproviding multiple resolution compression in accordance with the presentinvention.

FIG. 5 shows a motion estimation strategy according to an embodiment ofthe present invention.

FIG. 6 shows a non-linear video editor according to an embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention is illustrated herein using an exemplaryimplementation of a multiple resolution video compression systemparticularly well-suited for use in applications such as “non-linear”video editors and video servers. It should be understood that thetechniques of the present invention are more broadly applicable to anyapplication in which it is desirable to generate multiple resolutionversions of a given video signal. The invention is suitable for use withMPEG-2 encoding as well as other video encoding standards. The term“reduced resolution” as used herein is intended to include any versionof a given video signal which has a lower resolution than an unscaledversion of the given signal supplied from a video source. The reducedresolution may be a reduced luminance resolution or a reducedchrominance resolution or both.

FIG. 4 shows an exemplary embodiment of a multiple resolution videocompression system 30 in accordance with the present invention. A videosource 32 supplies a video signal to a first video compressor 34 and avideo scaler 36. The video source 32 may be a single video signal sourcesuch as a video camera or video cassette recorder or a multiple-signalsource made up of a combination of various sources. The video scaler 36scales the video signal to at least one reduced resolution and sends theresulting reduced resolution version or versions to a second videocompressor 38. The first video compressor 34 compresses the fullresolution unscaled version of the video signal from source 32 andstores the resulting compressed video bit stream in the storage device40. The storage device 40 may be an optical or magnetic disk drive ordisk array, an electronic memory or other suitable compressed videostorage device. The second video compressor 38 compresses at least onereduced resolution version of the video signal to generate at least onecorresponding reduced resolution compressed video bit stream. Thecompressed stream or streams from the second video compressor 38 arealso stored in the storage device 40. The compressed bit streamsgenerated by the first and second video compressors 34, 38 may differ inbit rate, group of picture (GOP) structure and/or other parameters. Thevideo scaler 36 and the video compressors 34, 38 may be combined into asingle device to facilitate sharing of hardware resources as will bedescribed in greater detail below.

At least one reduced resolution version of the video signal is suppliedfrom the video scaler 36 via line 44 to the first video compressor 34.The reduced resolution version is used in a hierarchical motionestimation (ME) search performed on frames of the unscaled fullresolution video signal in the first video compressor 34. The scaler 36and first video compressor 34 can therefore share at least a portion ofa common memory. The first video compressor 34 performs the hierarchicalME search and generates ME search results which are supplied via line 46to the second video compressor 38. The second video compressor 38 usesthe ME search results from the first video compressor 34 in encoding thereduced resolution version supplied from the video scaler 36. Forexample, the second video compressor 38 may utilize an average of motionvectors supplied from the first video compressor 34 or the actual motionvectors generated by a particular hierarchical ME search stage. One wayin which the second video compressor 38 can use the results of one stageof the motion estimation is shown in FIG. 5. For this example, it isassumed that the second video compressor 38 compresses video at ¼ theresolution of the first video compressor 34. The full resolution videosignal is decimated to ¼ and {fraction (1/16)} resolutions. Hierarchicalmotion estimation is then used in compressing the video signals. In sucha motion estimation, a zero centered search is conducted on the{fraction (1/16)} resolution image. The results of the first search areused as a basis to perform a second search on the ¼ resolution image(e.g., in the local vicinity of one or more candidate motion vectorsidentified in the first search). The results of the second search maythen be used as a basis for performing a search on the full resolutionimage. The results of the second search may also be, used directly togenerate motion vectors in the second video compressor 38. The resultsof the third search on the full resolution image are used to generatemotion vectors for the first video compressor 34. The first and secondcompressors 34, 38 may share common video compression hardware. Forexample, a single discrete cosine transform (DCT) unit could be timemultiplexed between the first and second compressors 34, 38. Other typesof video compression hardware which could be time multiplexed orotherwise shared between first and second compressors 34, 38 include theinverse DCT, quantizer, inverse quantizer, motion compensator, run-levelencoder, variable length encoder and pre-processing filters.

An exemplary hierarchical ME technique suitable for use in the system 30of FIG. 4 is described in U.S. patent application Ser. No. 08/652,231 ofJohn Ju filed on May 23, 1996 and entitled “Video Encoding WithMulti-Stage Projection Motion Estimation,” which is assigned to thepresent assignee and incorporated by reference herein. In accordancewith that technique, the first video compressor 34 of system 30 mayreceive a video signal from source 32 in the form of a sequence ofCCIR601 video images. For each CCIR601 image, the video scaler 36generates first, second and third reduced resolution images which may bea QQCIF image, a QCIF image and a CIF image, respectively, which are{fraction (1/64)} size, {fraction (1/16)} size and ¼ size, respectively,relative to the CCIR601 image. These reduced resolution images aresupplied via line 44 to the first video compressor 34 as previouslydescribed. Alternatively, the video scaler 36 may generate only a subsetof the reduced resolution versions of the signal from source 12, withthe remaining reduced resolution versions generated within the firstvideo compressor 34.

The video compressor 34 performs a first stage motion vector search onthe {fraction (1/64)} size QQCIF image using a (0,0) motion vectorstarting point and a first search range suitable for detecting globalmotion within the original CCIR601 image. Global motion refers generallyto motion involving many objects in a given image, such as the motionproduced by a panning or zooming operation. The first video compressor34 then performs a second stage motion vector search on the {fraction(1/16)} size QCIF image using the (0,0) starting point and a secondsmaller search range suitable for detecting local motion within theoriginal CCIR601 image. Local motion refers generally to motioninvolving small and/or irregular movements of a particular object orobjects in a given image. The first video compressor 34 then performs athird stage motion vector search on the ¼ size CIF image using astarting point based on scaled versions of the motion vectors identifiedin the first and second stage searches, and a search range smaller thanthe first and second search ranges. The first video compressor 34finally performs a fourth stage search on the original CCIR601 image ora reconstruction thereof using the motion vectors identified in thethird search stage. A motion compensation type and a suitable set ofmotion vectors for the original CCIR601 image is determined based on theresults of the fourth stage search. This motion compensation type andset of motion vectors may be supplied to the second video compressor 38via line 46 and used in the encoding of one or more reduced resolutionimages. It should be noted that the present invention may utilize any ofa number of other types of hierarchical ME search techniques.

The video compression system 30 of FIG. 4 is well-suited for use invideo server applications. For example, a video server may utilize thefirst video compressor 34 to generlte a high resolution, high bit rate,inter-frame encoded version of a given video signal for delivery overrelatively high bandwidth ISDN or T1 connections, while the second videocompressor 38 is used to generate a low resolution, low bit rate,intra-frame encoded version of the given video signal for delivery overrelatively low bandwidth POTS connections.

The video compression system 30 of FIG. 4 is also well-suited for use innon-linear video to editor applications. For example, a high resolutioncompressed bit stream generated by the first video compressor 34 andstored in storage device 40 could be used in the final edited video,with one or more reduced resolution bit streams generated by the secondvideo compressor 38 used to provide trick mode playback, fast randomaccess, multiple simultaneous stream playback and other functions whichfacilitate the editing process.

FIG. 6 shows a non-linear video editing system 60 according to anotherembodiment of the present invention. As shown, a linear-video source 62outputs an original video signal to a first video compressor 64 and asecond video compressor 68. The first video compressor 64 produces ahigh quality compressed video bitstream and the second video compressor68 produces a low quality compressed video bitstream. For purposes ofthis specification, a low-quality compressed bit stream is created byeither compressing a version of the video signal that is of lowerquality than the version of the video signal compressed by the secondvideo compressor 68 or compressing a video signal using a less efficientcompression technique. For example, the version of the video signalreceived at the first video compressor 64 may have a lower spatialresolution, a lower temporal resolution, or a lower bit rate, than theversion of the original video signal received at the second videocompressor. Likewise, the version of the original video signal receivedat the first video compressor 64 may be noninterlaced (i.e., progressivescanned or de-interlaced) whereas the version of the video signalreceived at the second video compressor 68 may be interlaced. An exampleof such a situation is where the version of the video signal received atthe first video compressor 64 is SIF (352×240×29.97 frames per second)format and the version of the original video signal received at thesecond video compressor 68 is HHR (352×240×59.98 fields per second)format. Another example is where the first version of the video signal(fed to compressor 68) has a bit rate of 5 mbits/sec and where thesecond version of the video signal (fed to compressor 64) has a bit rateof 30 mbits/sec.

To produce a spatially or temporally scaled video bit stream for inputto the first video compressor 64, the original video signal may be firstscaled by scaler 63. The scaled video signal is inputted to the firstvideo compressor 64 but the unscaled original video signal is inputtedto fit the second video compressor 68. Note, then, that the use of avideo scaler 63 is only illustrative of the type of device which can beused to lower the quality of the compressed video signal produced by thefirst video compressor 64. Video scaler 63 may be replaced with ade-interlacer, or some combination of the de-interlacer and the videoscaler.

Alternatively, the video compressors 64 and 68 receive the same versionof the input video signal, in which case, the video scaler 63 is absent.Rather, the video compressor 68 produces a lower quality compressedvideo signal because the video compressor 68 operates according to aless efficient compression standard (e.g. MPEG-1, motion JPEG, H.261,H.263), uses fewer compression options (i.e., users intra-coding only,uses frame prediction only, uses a smaller motion vector search range,etc.) or simply compresses the video to a lower bit rate than the secondvideo compressor 64. Alternatively, the video compressors 64 and 68 bothreceive different quality video signals and compress the video signalsusing different efficiency compression techniques.

Despite these differences between the low and high quality compressedvideo bit stream, the compressors 64 and 68 may optionally sharecompression results to,improve performance. Nevertheless, each of thelow quality and high quality compressed video bit streams isindependently decompressible, i.e., without reliance on any informationin the other compressed video bit stream. The high quality compressedversion of the video signal is stored on a first non-linear storagedevice 66. The low quality compressed version of the video signal isstored on a second non-linear storage device 66′. Preferably, the firstand second non-linear storage devices 66 and 66′ are a single non-linearstorage device. Also provided are a computer 72 for viewing and editingvideo signals and a decompressor 69 for decompressing the low and highquality compressed video signals stored in the non-linear storagedevices 66 and 66′. The computer 72 in FIG. 6 may be embodied as ageneral purpose workstation or a general purpose work station withadditional hardware for performing video effects. The video compressors64 and 68 and decompressor 69 may be embodied as discrete integratedcircuits or as a single integrated circuit. The additional hardware forperforming video effects may be embodied on the same integrated circuitas the video effects may be embodied on the same integrated circuit asthe video decompressor 69.

In operation, the operator views and edits the video as decompressedfrom the low quality compressed video signal. The operator may previewedits as performed on the low quality compressed video signal or thehigh quality compressed video signal. Preferably, however, the formationof the edited video event is deferred. Instead, a video edit decisionlist is constructed during the course of editing the video signals.After the operator is satisfied with the edit decisions, the highquality compressed video signal is edited according to the edit decisionlist. Afterwards, the edited video event in high quality compressed formis stored on the non-linear storage device 66. This edited video eventillustratively may be decompressed by decompressor 69 and transmitted orstored using on a linear storage device such as VTR 62.

In general, the low quality compressed video signal is easier to store,retrieve, decompress and manipulate (perform edit operations on) thanthe high quality compressed video signal. For example, it may be easierto play back simultaneously a large number of low quality (low bit rate)bit streams, to play back such bit streams in reverse order, to playback such bit streams in fast forward or fast-reverse, etc. It is alsopossible that the non-linear storage device 66, computer 72 and/ordecompressor 69 can only accommodate such operations/transitions on thehigh quality compressed video signal in non-real time. Note that it ispossible to provide equipment which can support such real timeoperations and transitions on the high quality compressed video signalas well as the low quality compressed video signal. However, a savingscan be achieved since less expensive/sophisticated non-linear storagedevices 66 and 66′, decompressors 69 and computers 72 can be used whichneed not support real-time operations/transitions on the high qualitycompressed video signal.

The video editor 60 provides several editing options not available inthe conventional online video editor 10 of FIG. 1. For instance, theoperator can view video clips at a fast-forward or fast-reverse speedeven though the equipment (e.g., non-linear storage device 66′,decompressor 69, computer 72) might only accommodate such real-timeoperations for the low quality compressed video signal (and therefore issubstantially less costly than equipment that can also perform suchoperations/transitions on the high quality compressed video signal). Forthe same reason, the operator can use substantially less expensiveequipment to view multiple clips simultaneously.

Note also that if the low quality compressed video signal usesintra-frame coding whereas a high quality compressed video signal usesboth intra-frame and inter-frame coding, then reverse direction viewingof low quality clips is substantially simpler. (This is becauseinter-frame encoded video includes pictures that are encoded using datain preceding and/or succeeding reference pictures as predictors. Suchreference pictures may be located in the bit stream of the high qualitycompressed video signal in a different order than the order in whichthey are presented or displayed to facilitate forward directiondecompression and presentation. However, this reordering in the bitstream renders reverse order decompression and presentation moredifficult.) Decompressors 69 which support reverse order play back ofinter-frame encoded video signals are substantially more complex andexpensive than decompressors that do not provide such a feature forinter-frame encoded video. Thus, the editor 60 provides a low-costsolution for accommodating real-time reverse play back of video clips(albeit, using only the low quality compressed video signal).

Furthermore, note that the operator can easily preview several complextransitions on the low quality compressed video signal to determinewhich produces the best results. Such complex transitions might bepossible in real-time, or in any event, using less time than performingsuch transitions on the high quality compressed video signal. In onemanner of preview, the operator performs several real-time complextransitions on the low quality compressed video signal. The operatorselects a desired complex transition and performs such a transition onthe high quality compressed video signal. Illustratively, the equipmentcan only perform the complex transition on the high quality video signalin non-real-time. The computer 72 signals the storage device 66 toretrieve the appropriate portion(s) of the high-quality video signal andsignals the decompressor 69 to decompress them. If necessary, thecomputer 72 performs the complex transition on the decompressed videosignal. The edited video signal with the complex transition thusproduced is compressed (using the high quality compressor 64) and storedon the non-linear storage device 66.

More generally stated, decompressor 69 of the video editor 60 may beable to decompress, simultaneously, in real-time, N low qualitycompressed bit streams (for simultaneous display), but to decompresssimultaneously, in real-time, only M high quality compressed bit streams(for simultaneous display), where N>M≧0. Likewise, the computer 72 andcompressor 68 may be able to edit simultaneously, in particular, tosimultaneously perform effects on, N low quality compressed bit streams,whereas the computer 72 and compressor 64 may be able to only editsimultaneously M high quality compressed bit streams, where N>M≧0. Forexample, it is possible that N=3 low quality compressed bit streamscould be viewed in real-time simultaneously, but only M=1 high qualitycompressed bit stream can be viewed in real-time. Alternatively, it ispossible to edit/perform effects on N=2 low quality compressed bitstreams in real-time, e.g., a dissolve of one low quality compressed bitstream into another, yet only be able to perform such effects on thehigh quality compressed bit stream in non-real-time. In yet anotherembodiment, the editor 60 can perform effects on N=3 low qualitycompressed bit streams in real-time, such as a dissolve of one lowquality compressed bit stream into a second low quality compressed bitstream, with a third low quality compressed bit stream in a small“picture-in-picture” box. However, the editor 60 might only be able toperform, effects on M=2 high quality compressed bit streams inreal-time, e.g., the dissolve of a first high quality compressedbitstream into a second high quality compressed bitstream, but notsimultaneously also provide a third high quality compressed bit streamin a picture-in-picture box. Each of these embodiments is desirablesince more effects would be necessary for previewing, in low quality,edits/effects before finalizing, and performing, the finalizededit/effect in high quality.

The description of the architecture above assumes that the equipment(e.g., decompressor 69, non-linear storage devices 66 and 66′ andcomputer 72) support full real time operations/transitions on lowquality compressed video signals but only non-real timetransitions/operations on the high quality compressed video signals. Ofcourse, the same architecture can be used with more expensive equipmentthat can support all transitions/operations on the high qualitycompressed video signal.

The foregoing description is merely illustrative of the invention. Thosehaving ordinary skill in the art may devise numerous alternativeembodiments within the scope of the appended claims. For example,alternative embodiments of the invention may utilize any number ofadditional video sources 32, video scalers 36 and video compressors 34,38 arranged to process any number of video signals to generate anynumber of corresponding compressed video bit streams. The compressed bitstreams may be stored on a single storage device 40 as shown in FIG. 3or on multiple storage devices.

The claimed invention is:
 1. A nonlinear editor comprising: a firstvideo compressor for compressing a first version of an input videosignal to produce a low quality compressed video signal, and a secondvideo compressor for compressing a second version of said input videosignal to produce a high quality compressed video signal, said highquality compressed video signal having a higher quality than said lowquality compressed video signal, wherein each of said low quality andhigh quality compressed video signals produced by said first and secondvideo compressors, respectively, are independently decompressible,wherein said first version of said video signal is noninterlaced andsaid second version of said video signal is interlaced.
 2. The editor ofclaim 1 further comprising: a video scaler receiving said second versionof said input video signal and producing said first version of saidvideo signal, said first version of said video signal having a lowerspatial resolution as said second video signal.
 3. The editor of claim 1wherein said first compressor compresses said first version of saidinput video signal using a different compression standard as used bysaid second compressor in compressing said second version of said inputvideo signal.
 4. The editor of claim 1 wherein said first compressorcompresses said first version of said input video signal at a lower bitrate than said second compressor compresses said second version of saidinput video signal.
 5. A nonlinear editor comprising: a first videocompressor for compressing a first version of an input video signal toproduce a low quality compressed video signal, a second video compessorfor compressing a second version of said input video signal to produce ahigh quality compressed video signal, said high guality compressed videosignal having a higher quality than said low quality compressed videosignal, and a de-interlacer receiving said second version of said inputvideo signal and producing said first version of said input videosignal, said first version of said video signal comprising frames formedfrom fields of said second version of said input video signal, whereineach of said low quality and high qualily compressed video signalsproduced by said first and second video compressors, respectively, areindependently decompressible.
 6. A nonlinear editor comprising: a firstvideo compressor for compressing a first version of an input videosignal to produce a low quality compressed video signal, and a secondvideo compressor for compressing a second version of said input videosignal to produce a high quality compressed video signal, said highquality compressed video signal having higher quality than said lowguality compressed video signal, wherein each of said low quality andhigh quality compressed video signals produced by said first and secondvideo compressors, respectively, are independently decompressible,wherein said first compressor compresses said first version of saidinput video signal using a different compression standard as used bysaid second compressor in compressing said second version of said inputvideo signal, and wherein said first and second versions of said inputvideo signal are identical.
 7. A nonlinear editor comprising: a firstvideo compressor for compressing a first version of an input videosignal to produce a low quality compressed video signal, and a secondvideo compressor for compressing a second version of said input videosignal to produce a high guality compressed video signal, said highquality compressed video signal having a higher quality than said lowquality compressed video signal, wherein each of said low quality andhigh quality compressed video signals produced by said first and secondvideo compressors, respectively, are independently decompressible, andwherein said first video compressor compresses said first version ofsaid input video signal using only intra coding and wherein said secondcompressor compresses said second version of said input video signalusing both intra coding and inter coding.
 8. A nonlinear editorcomprising: a first video compressor for compressing a first version ofan input video signal to produce a low quality compressed video signal,a second video compressor for compressing a second version of said inputvideo signal to produce a high quality compressed video signal, saidhigh quality compressed video signal having a higher quality than saidlow quality compressed video signal, and a decoder capable ofsimultaneously presenting N of said low quality compressed video signalsin real time and capable of simultaneously presenting M of said highquality compressed video signals in real time, where N>M>=0, whereineach of said low quality and high quality compressed video signalsproduced by said first and second video compressors, respectively, areindependently decompressible.
 9. The editor of claim 8 furthercomprising: a computer for enabling construction of an edit decisionlist for editing one or more of said M high quality compressed videosignals based on said presentation of said N low quality compressedvideo signals.
 10. The editor of claim 9 wherein said computer editssaid one or more high quality compressed video signals according to saidedit decision list.
 11. A nonlinear editor comprising: a first videocompressor for compressing a first version of an input video signal toproduce a low quality compressed video signal, a second video compressorfor compressing a second version of said input video signal to produce ahigh quality compressed video signal, said high quality compressed videosignal having a higher quality than said low quality compressed videosignal, and a computer capable of performing effects on N decoded onesof said low quality video signals in real time and capable of performingeffects on M decoded ones of said high quality video signals in realtime, where N>M>=0, wherein each of said low quality and high qualitycompressed video signals produced by said first and second videocompressors, respectively, are independently decompressible.
 12. Theeditor of claim 11 wherein said computer constructs an edit decisionlist based on effects performed on up to N of said decoded low qualityvideo signals and performs effects from said edit decision list on up toM of said decoded high quality video signals.
 13. A method for editing avideo signal comprising the steps of: compressing a first version of aninput video signal to produce a low quality compressed video signal, andcompressing a second version of said input video signal to produce ahigh quality compressed video signal, said high quality compressed videosignal having a higher quality a said low quality compressed videosignal, wherein each of said low quality and high quality compressedvideo signals produced by said steps of compressing are independentlydecompressible, and wherein said first version of said video signal isnoninterlaced and said version of said video signal is interlaced. 14.The method of editing of claim 13 further comprising: scaling saidsecond version of said input video signal to produce said first versionof said video signal so that said first version of said video signal hasa lower resolution as said second video signal.
 15. The method ofediting of claim 13 further comprising: compressing said first versionof said input video signal using a different compression standard asused to compress said second version of said input signal.
 16. Themethod of editing of claim 13 wherein said low quality compressed videosignal has a lower bit rate than said high quality compressed videosignal.
 17. A method for editing a video signal comprising the steps of:compressing a first version of an input video signal to produce a lowquality compressed video signal, compressing a second version of saidinput video signal to produce a high quality compressed video signal,said high quality compressed video signal having a higher quality thansaid low quality compressed video signal, and de-interlacing said secondversion of said input video signal to produce said first version of saidinput video signal so that said first version of said video signalcomprises frames formed from fields of said second version of said inputvideo signal, wherein each of said low quality and high qualitycompressed video signals produced by said steps of compressing areindependently decompressible.
 18. A method for editing a video signalcomprising the steps of: compressing a first version of an input videosignal to produce a low quality compressed video signal, and compressinga second version of said input video signal to produce a high qualitycompressed video signal, said high guality compressed video signalhaving a higher quality than said low quality compressed video signal,wherein each of said low quality and high quality compressed videosignal produced by said steps of compressing are independentlydecompressible, and wherein said first and second versions of said inputvideo signal are identical.
 19. A method for editing a video signalcomprising the steps of: compressing a first version of an input videosignal to produce a low quality compressed video signal, compressing asecond version of said input video signal to produce a high qualitycompressed video signal, said high qualily compressed video signalhaving a high quality than said low quality compressed video signal,compressing said first version of said input video signal using onlyintra coding, and compressing said second version of said input videosignal using both intra coding and inter coding, wherein each of saidlow quality and high quality compressed video signals produced by saidsteps of compressing are independently decompressible.
 20. A method forediting a video signal comprising the steps of: compressing a firstversion of an input video signal to produce a low quality compressedvideo signal, compressing a second version of said input video signal toproduce a high quality compressed video signal, said high qualitycompressed video signal having a higher quality than said low qualitycompressed video signal, simultaneously presenting N of said low qualitycompressed video signals in real time, and simultaneously presenting Mof said high quality compressed video signals in real time, whereN>M>=0, wherein each of said low quality and high quality compressedvideo signals produced by said steps of compressing are independentlydecompressible.
 21. A method for editing a video signal comprising thesteps of: compressing a first version of an input video signal toproduce a low quality compressed video signal, compressing a secondversion of said input video signal to produce a high quality compressedvideo signal, said high quality compressed video signal having a higherquality than said low quality compressed video signal, and performingeffects on N decoded ones of said low quality video signals in real timeand performing effects on M decoded ones of said high quality videosignals in real time, where N>M>=0, wherein each of aid low quality andhigh quality compressed video signals produced by said steps ofcompressing are independently decompressible.
 22. A method for editing avideo signal comprising the steps of: compressing a first version of aninput video signal to use a low quality compressed video signal,compressing a second version of said input video signal to produce ahigh quality compressed video signal, said high quality compressed videosignal having a higher quality than said low quality compressed videosignal, and constructing an edit decision list based on effectsperformed on up to N of said decoded low quality video signals andperforming said same effects on up to N of said decoded high qualityvideo signals, wherein each of said low quality and high qualitycompressed video signals produced by said steps of compressing areindependently decompressible.
 23. A method for editing a video signalcomprising the steps of: compressing a first version of an input videosignal to produce a low quality compressed video signal, compressing asecond version of said input video signal to produce a high qualitycompressed video signal, said high quality compressed video signalhaving a higher quality than said low quality compressed video signal,and enabling construction of an edit decision list for editing one ormore of said M high quality compressed video signals based on saidpresentation of said low quality compressed video signals, wherein eachof said low quality and high quality compressed video signals producedby said steps of compressing are independently decompressible.
 24. Themethod of editing of claim 23 further comprising the step of: editingsaid one or more high quality compressed video signals according to saidedit decision list.